Support and Efficiency of Nested Parallelism in OpenMP Implementations

Nested parallelism has been a major feature of OpenMP since its very beginnings. As a programming style, it provides an elegant solution for a wide class of parallel applications, with the potential to achieve substantial utilization of the available computational resources, in situations where outer-loop parallelism simply can not. Notwithstanding its significance, nested parallelism support was slow to find its way into OpenMP implementations, commercial and research ones alike. Even nowadays, the level of support is varying greatly among compilers and runtime systems. In this work, we take a closer look at OpenMP implementations with respect to their level of support for nested parallelism. We classify them into three broad categories: those that provide full support, those that provide partial support and those that provide no support at all. The systems surveyed include commercial and research ones. Additionally, we proceed to quantify the efficiency of the implementation. With a representative set of compilers that provide adequate support, we perform a comparative performance evaluation. We evaluate both the incurred overheads and their overall behavior, using microbenchmarks and a full-fledged application. The results are interesting because they show that full support of nested parallelism does not necessarily guarantee scalable performance. Among our findings is the fact that most compilers do not seem to handle nested parallelism in a predictable and stable way as the number of threads increases beyond the system’s processor count.